Learning to provide appropriate assistance to people indifferent situations is an extremely important, but insufficientlyinvestigated machine learning task. Applications includehuman-robot and human-computer interactions settings to maximizing the benefits of assistive technologies. Three key challenges must be overcome to appropriately address this task: Complexity: the space of possible assistive policies can be very large, making many existing methods (e.g., fromreinforcement learning) too data inefficient to be practical. Noise and misspecification: observed human behavior is often noisy and parametric formulations that reduce complexity will typically suffer from model misspecification,leading to unboundedly sub-optimal assistance. Biasedness: data available for learning a model is biased by previously provided assistive actions, violating the typical assumptions of supervised learning. We develop a general framework for learning to assist in single intervention settings. The framework narrows the search for effective assistance by viewing previous behavior under assistance through a restricted set of statistics. Assistive policies for the worst-case context-assistance-outcome relationships satisfying these statistics are obtained. We embed the problem of learning how to assist users in cursor based target pointing tasks into this framework and outline its usage.